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Dear friends, 


I do not believe in having a perfect manual that can support for 
participatory exercises. Participatory evaluation process must be 
flexible and appropriate to the local context. However, a reference 
guideline can be used as and when necessary. Practical experiences 
can make one effective to facilitate participatory evaluation process. 

I, therefore, expect this handout can be helpful and should be used as 
a reference guideline only. 

Thank you. 

Kamal Phuyal 
October 2012 




Contents 


1. What is participatory evaluation? 7 

2. Participatory and conventional evaluation: some differences 11 

3. A framework for participatory evaluation? 12 

4. Why participatory evaluation is done 13 

5. Types of indicators 14 

6. Stages of participatory evaluation 17 

7. Indicators to be used and checklist required 19 

8. Tools to be used 21 

9. Indicators-checklist-tools for participatory evaluation 22 

10. Potential challenges and suggestions for solution 24 

11. Role of the facilitators 25 

12. Necessary qualities of the facilitators 26 

13. Some visual clips of participatory exercises 29 

14 References 31 




1. What is participatory evaluation? 


In recent years, participatory evaluation concept has been attracting 
growing attention among the development professionals both in 
government and non-governmental sectors. However, a single 
definition cannot represent the wider meaning of participatory 
evaluation. 

Participatory evaluation emphasizes the importance of the people' 
participation in the development process and raises their ownership 
feeling. It involves various stakeholders and partners 1 in the collective 
examination and assessment of those programmes or projects they are 
involved in. As R. Thompson writes (Participatory development 
documents, 1995) participatory evaluation is based on the assertion 
that evaluation goes beyond science, getting the facts, to include the 
human, political, social, cultural, and contextual elements. It is people 
centred: organization and local people are the key actors of the 
evaluation process and not the mere objects of the evaluation. 

Participatory evaluation facilitates local people to improve their 
efficiency and effectiveness. It is also an educational process in which 
participants increase awareness and understanding of the various 
factors, which affect them, thereby increasing their control over the 
development process. 

Various stakeholders with active participation of the local people 
carry out participatory evaluation jointly. Evaluation is also like to see 
where and how fast the programme is going, and then estimating 
when it is likely to reach its destination. Participatory evaluation, 
however, focuses to let the local people to express their perceptions in 
order to monitor progress and effectiveness, consider costs and 
efficiency, show where changes were needed, and help to plan more 
effectively for the future. 2 


1 Many people refer the term beneficiaries, but, I prefer to use the term partners. 

2 M. T. Feuerstein, 1993, Partners in Evaluation. 
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Some descriptions about participatory evaluation are as following. 


Deepa Narayan, 1996 

Participatory evaluation is a process of collaborative problem¬ 
solving through the generation and use of knowledge. It is a 
process that leads to corrective action by involving all levels of 
users in shared decision making. 

Hope, A., Timmel, A., 1996, Training for Transformation Part III 
Participatory evaluation is a process of involving participants in 
programmes to reflect critically on their own projects, 
programmes, aims and leadership. It is participant-centred. It is an 
important part of the development education process. Its value is 
that it continues the process of action-reflection, and increases the 
awareness that people themselves can shape their own lives and 
destiny. 

Carole C. Upshur, 1995 

Participatory evaluation (PE) is a method of inquiry in the family 
of participatory and action research. These traditions in research 
and evaluation grew out of conflicts and contradictions about how 
knowledge is created and used. 

USAID, 1996, Participatory evaluation and international cooperation, 

June 2001 

Participatory evaluation provides the opportunities of the active 
involvement in the evaluation process for those with a stake in the 
programme; providers, partners, customers (beneficiaries) and any 
other interested parties. Participation typically takes place in all 
phases of the evaluation: planning and designing of the evaluation; 
gathering and analysing the relevant data; identifying the 
evaluation findings, conclusions, and recommendation; 


Reference materials: participatory evaluation 


8 




disseminating results; and preparing an action plan to improve 
programme performance. 

OESP handbook, 1997 

Participatory evaluation, a dimension of participatory development 
embodying many of the same concepts, involves the stakeholders 
and beneficiaries of a programme or project in the collective 
examination and assessment of that programme or project. It is 
people centred: project stakeholders and beneficiaries are the key 
actors of the evaluation process and not the mere objects of the 
evaluation. 

UNDP, 1997 

Participatory evaluation is people centred: project stakeholders and 
beneficiaries are the key actors of the evaluation process and not 
the mere target of the evaluation. 

Innovative Network, Inc., 2003 

Participatory evaluation is the process of engaging stakeholders in 
an evaluation effort. (Typical stakeholders - the people most 
invested in the success of a program - include staff, board 
members, volunteers, sister agencies, and funder.) Getting input 
from your stakeholders at all stages of your evaluation effort - from 
deciding what questions to ask, to collecting data, to analyzing data 
and presenting results - is critical to the usefulness and ultimate 
value of the evaluation. A participatory approach to evaluation gets 
the "buy-in" of as many stakeholders as possible, so they have a 
feeling of ownership over evaluation results. This helps to create an 
atmosphere where people want to learn about how and why 
programs are achieving results. 

World Bank, 2000 

Participatory evaluation is a joint task process where stakeholders 
of various levels jointly evaluate the project and implement the 
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necessary revision measures. In this collaborative resolution of 
problems, men and women of communities, NGOs and other 
mediating organizations, private corporations involved in the 
project, and government employees of all levels are invited to 
participate. 

Evaluation guidelines, 1999, Danida 

An evaluation is an assessment, as systematic and objective as 
possible, of on-going or completed aid activities, their design, 
implementation and results. The aim is to determine the relevance 
and fulfilment of objectives, developmental efficiency, 
effectiveness, impact and sustainability. 
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2. Participatory evaluation and conventional evaluation: some 
differences 


Evaluation through 
conventional survey 

Participatory evaluation 

Emphasis on quantity 

Emphasis on quality 

Based on multi-topic survey 
with help of questionnaire 

Based on group interactions 
through participatory methods in 
different locations 

Whose reality? Outsiders play 
the role of experts. 

People's reality with people as 
analyst. 

Explain complex things in their 
way and with their emphasis. 

Community as beneficiaries 

Community as partners 

Handout mentality 

Community capacity 
enhancement 

Closed-ended inquiry 
Questionnaires fielded by field 
investigators 

Open-ended inquiry 

Field based participatory 
exercises 

Top-down process 

All questions cannot be 
anticipated 

Bottom-up process 

Poor people less involved in the 
process 

Poor people identified and their 
active participation 

'We change community' - 
attitude among the outsiders 

We facilitate so communities will 
change themselves - attitude 

Looking for 'average' picture and 
solutions 

Looking for diversity 
differential pictures and solutions 

Emphasizes quantitative aspects 

Has both qualitative and 
quantitative aspects 

Individual as unit with little 
scope for group work 

Group analysis and group 
presentation 

Tends to overlook gender 
aspects 

Gender aspect is important 
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3. Why participatory evaluation (A framework to understand the objectives of participator)' evaluation) 
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4. Why participatory evaluation is done 


Participatory evaluation is done not just for the sake of doing 
evaluation. Instead, it should focus more on the learning process. Its 
outcomes are not mere a descriptions of the project and its situation. 
Rather the evaluation process represents meaningful constructions of 
actors’ attempts to understand the situations in which they act. The 
evaluation facilitators therefore assist the local people to construct a 
shared reality about the project and to make collective judgements 
about the project accomplished and problems faced. 


to measure 

to see what has P ro § ress 

been achieved I 


to see where 
strengths and 
weaknesses lie 



Why 

to criticise our _^ 

own work evaluation 


to be able to 
improve monitoring 
methods 



to help to make 
better plans for the 
future 


to make our work 
more effective 


to compare the 
programmes with 
other like it 



to be able to share 
our experiences 


\ 

\ to compare between 
the achievement and 
costs 

to see all the 
effort has been 
effective 



'Primary stakeholders’ participation is essential in assessing and 
determining how to improve and promote sustainable 
development. Only participatory evaluation can ensure various 
stakeholders work actively together' (Okley, 1996) in determining 
and controlling what is to be evaluated, the evaluation indicators 
to be used, the instruments for measuring the chosen indicators 
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and the means of analysing interpreting, communicating and 
utilising the findings (O. Okuga, 2002). 

5. Types of indicators 

What is the basis of evaluation or what to evaluate has been very 
cmcial question in participatory evaluation. Indicators should be set in 
the beginning of the evaluation process. However, local people, the 
primary stakeholders, should participate in deciding indicators. The 
possible indicators can be described as follows: 


Indicator/s of 
availability 


Indicator/s of 
efficiency 


Indicator/s of 
coverage 


Indicator/s of 
accessibility 


Indicator/s of 
quality 




Indicator/s of 
process 


Indicator/s of 
equity 


Indicator/s of 
effort 


Indicator/s of 
impact 


Indicator/s of 
utilisation 


Indicator/s of 
relevance 


Indicator/s of 
sustainability 


Indicator/s of availability 

• It shows whether something exists or available in the 
community or not. 
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Indicator/s of efficiency 

• It shows whether resources and activities are being put to the 
best possible use to achieve the objectives, such as the number, 
frequency and quality. 

Indicator/s of process 

• It shows whether the planning and implementation is done 
properly or not. Who designed the project and how the project 
was implemented is analysed. 

Indicator/s of coverage 

• It shows what proportions of those who need something are 
receiving it. The number of people (or percentage) actually 
receiving the programmes benefit is analysed. 

Indicator/s of accessibility 

• It shows whether what exists is actually within reach of those 
who need it or not. 

Indicator/s of quality 

• It shows the quality or standard of the programmes. 

Indicator/s of utilisation 

• It shows to what extent programme that has been made 
available is being used for that purpose. 

Indicator/s of relevance 

• It shows how relevant or appropriate the programme is. 
Whether the programme done fits into local context is also 
analysed. 

Indicator/s of impact 

• It shows if what we are doing is really making any difference or 
not. What change occurred in the community is also analysed. 
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Indicator/s of effort 

• It shows how much and what is being invested in order to 
achieve the objectives. 

Indicator/s of equity 

• It shows whether the distribution of programmes was equitable 
or not. If there was any discrimination during the resource 
distribution or not is also analysed. 

Indicator/s of sustainability 

• It shows the strategy planned by the beneficiaries to continue 
the programmes in the future too. Capacity developed in the 
community is also analysed. 

Indicators should be finalised having discussions with various 
stakeholders, preferably with the local people before conducting 
evaluation exercises. Outsider facilitators and technical experts can 
help the local people to decide appropriate indicators for evaluation. 
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6. Stages of participatory evaluation 


Participatory evaluation process 


Indicators 



Note: This is a proposed evaluation process. Process can be followed in various 
ways, however, it should be people-centred, and should prefer to actively 
engage marginalised people. 
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Evaluation process: 


Collective 
decision for 
evaluation 


Evaluation 

committee 

formation 


Preparation for 
evaluation 


Evaluation 

exercises 


Sharing outcomes 
of evaluation 

Evaluation report 
preparation 


Use of report 


Implementing organization and local people should 
sit together, discuss and make decision for 
conducting evaluation. 

Local people's decision is most important for this. 

Evaluation committee is to be formed in order to 
coordinate/lead evaluation work. 

Local people should lead the committee. 
Representatives of implementing and other partners 
should stay in the committee. 

Local people need technical support for evaluation 
that can be provided by these representatives. 

A team can be formed to conduct evaluation 
exercises. 

Local youth can be involved in the team. 
Orientation on the concept of participatory 
evaluation to be done for team members. 

Training on using participatory tools should be 
given the team members. 

Other necessary preparation to be done: setting 
dates, times and venues with local people for 
evaluation exercises. 

Logistic preparation: arranging papers and other 
materials. 

Meet the people and share objectives. 

Conduct evaluation exercises using various 
participatory tools and techniques. 

Do analysis and draw results/learning. 

Share the results of evaluation. 

Listen to people how they react on the outcomes. 

Prepare a report of evaluation. Prepare first in local 
language and provide reports to the local people. 
Provide reports to all the stakeholders. 

Put efforts to the project for further improvement as 
per the suggestions made by the evaluation results. 
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7. Indicators to be used and checklist required 


Evaluation facilitators should develop a checklist, in coordination 
with local people and other stakeholders, of each indicator that can be 
used using various participatory tools. Some examples of the checklist 
have been shown below: 


Indicators to be used 

Checklist required 

Coverage 

• How many families have been covered by 
the project? 

• What area has been covered by the project? 

Accessibility 

• What is the process to participate the 
project? 

• Who have been able to join (and who are 
not) the project? 

• What efforts made to generate the 
accessibility of the marginalised people to 
the project? 

Process 

• How the process of the project was: 
designing to implementation level? 

• How local people participated in each 
stage? 

• What difficulties were faced - in which 
stages? 

Utilisation 

• Whether the learnt skills/knowledge have 
been utilised or not? If yes, by whom and 
how? 

Equity 

• Which people (from which well-being 
category) have participated in the project? 

• What about the participation of the weaker 
voices? 

• Who participate in the training (male or 
female)? 
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Quality 

• How is the quality of produced fishes? 
According to price received and according 
to the taste you have tested. 

Impact 

• What skills and knowledge have been 
gained? And how? 

• What Changes the project has brought? 

• What are the positive effects? 

• Is there any negative effects produced? 

• How the income has been used? 

Sustainability 

• How many members so far have been 
trained and on what? 

• What are the things 
(skills/knowledge/others) necessary for the 
effective continuation of the project? 


Note: Other necessary checklist will be added during the preparation 
of the workshop. 
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8. Tools to be used 


Various tools can be used for carrying out participatory evaluation. 
Facilitators can/should modify the application process of tools when 
necessary. Moreover, new tools can be generated. Here are some 
tools, which can be used during the participatory evaluation. 

1. Mapping tools 

a. Social and Resource mapping b. Mobility mapping 
c. Transect map/walk 

2. Diagramming tools 

a. Time line 
c. Trend diagram 
e. Daily routine diagram 

3. Ranking tools 

a. Pair-wise ranking 
c. Well-being ranking 

4. Discussion and interview tools 

a. Focus group discussions b. Semi-structured interview 

5. Other tools 

a. Pictures b. Spider-web diagram 


b. Seasonal calendar 
d. Flow diagram 
f. Impact diagram 


b. Direct matrix ranking 
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10. Potential challenges to be faced and suggestions for solution 


An absolute list of potential challenges cannot be finalised, which 
differs from one area to other, one context to other and also differs 
according to the time. Evaluation facilitators, therefore, should work 
out to be aware of challenges to be faced before conducting 
evaluation exercises. The following points give some ideas about the 
potential challenges (or risks) to the evaluation facilitators. 

Potential challenges to be faced: 

a. Discrimination/marginalized situation: 

• Women's low participation in front of men. 

• Children's low participation in front of adult. 

• Poor hesitate to speak out. 

• Illiterates feel difficult to express their feeling in front of 
educated people. 

b. Communication gap: 

• Language and other gaps (like level of awareness and way of 
thinking) might create difficulties for the local people to 
understand outsiders. 

• Outsiders are from different socio-cultural context, which might 
create gaps. 

• Biasness of the outsiders: seasonal, educational, and other 
biasness. 

c. Sensitive environment: 

• Discussions on sensitive (i.e. poverty, power, politics, etc.) 
issues might create some conflicting situation. 

• Exercises in public place might generate conflicting 
environment. 

• If the facilitators are not gender sensitive, conflict might be 
raised that affects the whole evaluation works. 
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d. Lack of skills and experiences: 

• Lack of experiences and skills among the facilitators might 
affect the evaluation exercises. 

e. Time factors: 

• Most 'participatory exercises' are conducted as per the 
convenient time of the organisations' staff members. In this 
situation, marginalized people (mainly, women, children and 
poor) might not be able to participate in the exercises as they 
are busy mainly during the day time. 

• Powerful (or elites) people of the community fix the time for 
the evaluation exercises. Marginalized people are often 
excluded while making such decisions. 

f. Rigidity: 

• Rigid evaluation approach (i.e. tools, methods, process) cannot 
encourage the local people to participate in the process. Rather, 
it excludes the voices of the poor, women and children. 

• Application of participatory tools cannot make the process 
participatory if the process itself is not flexible. 

g. Lack of immersion: 

• If adequate rapport with the local people is not developed, their 
inner (heartily) participatory is almost impossible. 

• Mostly during the workshop, this lacks as outsiders have less 
time and need to cover very many things. 

11. Role of the facilitators 

Evaluation team members have to play the following roles in order to 
make the evaluation process more participatory and bring the work 
into success. 
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a. Facilitation role: 

• Share the objectives of evaluation. 

• Discuss the importance of local people's participation in 
evaluation exercises and convince for their active participation. 

. Make the process easy or set the climate to support the local 
people to participate in the evaluation process. 

• Encourage the marginalized people to participate in the 
evaluation process. 

• Minimise the conflicting environment during the exercises. 

b. Documentation: 

• To document the information. 

• To make the notes of opinions expressed during the evaluation 
exercises. 


c. Observation: 

• Keen observation is necessary during the exercises in order to 
minimise the domination of powerful individuals or groups. 

• Handle the conflicting situation and manage the disturbing 
situation. 

A team of three facilitators has been found appropriate for effective 
facilitation for participatory exercises. However, it should not be 
taken as a formula. Three members can play the roles mentioned 
above more effectively. Team of female facilitators would be better to 
conduct evaluation exercises with women groups. 

12. Necessary qualities of the facilitators 

Facilitators should have the following qualities to facilitate 
participatory evaluation exercises for fruitful results. 
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a. Attitude and behaviour: 

• Respecting local culture and people. 

• Listening to the local people's ideas. 

• Flexible and friendly behaviour. 

• Have no biasness and should be positive. 

b. Concept/knowledge 

• Knowledge on participatory evaluation: what is participation, 
what is evaluation, etc. 

• Knowledge on social diversities. 

• Understand the power structure in the society and marginalized 
situation. 

• Clear ideas on and about the evaluation process and stages to be 
followed. 

• Know how and why conflicts are raised. 

• Teamwork. 

• Knowledge about the projects and & program 

• Knowledge about the organization (evaluation organizer) 


c. Skills 

• Facilitation skills. 

• Knowledge and experiences on using participatory tools. 

• Conflict management skills. 

• Group mobilisation skills. Etc. 
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13. Some visual clips of participatory exercises 
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